
Image Processing System Including Synchronous Type Processing Unit and 
Asynchronous Type Processing Unit, and Image Processing Method 

This application is based on Japanese Patent Application No. 10- 
5 317997 filed in Japan, the contents of which are hereby incorporated by 
reference. 

BACKGROUND OF THE INVENTION 
Field of the Invention 

The present invention relates to image processing systems, 

10 particularly to an image processing system including a synchronous type 
processing unit and an asynchronous type processing unit. 
Description of the Related Art 

A conventional image processing apparatus is known that reads out 
image data stored in a memory to carry out image processing on the read 

15 out image data. The image processing apparatus can be formed of a 

synchronous type or an asynchronous type circuit. A hardware circuit 
such as the ASIC (application specific integrated circuit) is exemplary of 
the synchronous type circuit. A circuit that carries out a process with 
software formed of a MPU (microprocessor) and the like is exemplary of the 

20 asynchronous type circuit. 

The conventional image processing apparatus imposes problems set 
forth in the following. 

In an image processing apparatus entirely formed of the synchronous 
type circuit, the processing speed is determined by the process of the heavy 

25 load. There was a problem that the process is slowed down even if it is not 
necessary to carry out a heavy load process. This is because a process 
corresponding to the time consumed for the heavy load process is required 
whatever image data is processed. For example, in a synchronous type 
circuit 900 shown in Fig. 11 including a line memory 901, a matrix 

30 processing unit 903 and a y correction unit 905, data delay occurs since 
image data of N lines must always be stored in line memory 901. Also, 
since a complicated matrix operation process (for example, integration of 
floating point) is carried out, data (operational result) delay occurs. 
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When the entirety is implemented by the synchronous type circuit, 
the synchronous circuit per se becomes complex and lacks flexibility. 
When the resolution or the like of the image data is altered, the entire 
structure of the apparatus must be modified, resulting in increase in cost. 

When the entirety of the image processing apparatus is implemented 
by the asynchronous type circuit, high speed processing is degraded. 
SUMMARY OF THE INVENTION 

In view of the foregoing, an object of the present invention is to 
provide an image processing apparatus that allows high speed processing, 
and that prevents increase in the cost when the resolution or the like is 
altered. 

According to an aspect of the present invention, an image processing 
system includes a synchronous type processing unit carrying out a first 
image process on image data that is the subject of processing, an 
15 asynchronous type processing unit carrying out a second image process on 
image data of a predetermined region out of the image data that is the 
subject of processing, and a synthesize unit synthesizing the output of the 
synchronous type processing unit with the output of the asynchronous type 
processing unit to form one image data. 
20 Preferably, the synthesize unit includes a memory storing the output 

of the synchronous type processing unit, and a replacement unit replacing a 
portion of the output of the synchronous type processing unit stored in the 
memory with the output of the asynchronous type processing unit. 
Further preferably, the image processing system includes a 
25 controller controlling the replacement timing by the replacement unit by 
detecting the state of the output of the synchronous type processing unit 
stored in the memory. 

The asynchronous type processing unit carries out a second image 
process only on image data of a predetermined region. Therefore, an 
image processing system that allows high speed processing is provided. 
When there is no predetermined region on which the second image process 
is to be carried out, only the output of the synchronous type processing unit 
is to be used. Therefore, high speed processing is allowed. 
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The asynchronous type processing unit has flexibility. Therefore, it 
is easy to modify only the processing contents that require modification (for 
example N-dimensional process) when the resolution or the like of the 
image data is altered. Thus, increase in the cost can be suppressed even 
5 in the case where the resolution or the like is altered. 

By carrying out the second image process through the asynchronous 
processing unit during the process through the synchronous type 
processing unit, the process can be further speeded. 

According to another aspect of the present invention, an image 
10 processing system includes a first image processor formed of a hardware 

circuit, and carrying out a first image process on input image data, a second 
image processor carrying out a second image process on a portion of the 
input image data according to a program of predetermined software, and a 
memory in which image data subjected to the first image process and the 
15 image data subjected to the second image process are synthesized and 
stored. 

Preferably in the image processing system, the data of the memory in 
which image data subjected to the first image process is stored is replaced 
with image data subjected to the second image process. 
20 Preferably, the second image processor detects the write timing of 

the image data subjected to the first image process into the memory to 
control the write timing into the memory. 

Preferably, the software is rewritable. 

Preferably, the second image processor detects the region of carrying 
25 out the second image process by scanning the input image data. 

According to a further aspect of the present invention, an image 
processing method includes the steps of carrying out a first image process 
on input image data through a hardware circuit, carrying out second image 
process on a portion of the input image data through software, and 
30 synthesizing image data subjected to the first image process with image 

data subjected to the second image process. The sequence of the first and 
second image processes is arbitrary. 

Preferably, the second image process step includes the step of 
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detecting the region of carrying out the second image process by scanning 
the input image data and carrying out the second image process on the 
detected region. 

The foregoing and other objects, features, aspects and advantages of 
5 the present invention will become more apparent from the following 

detailed description of the present invention when taken in conjunction 

with the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a diagram to describe an operation of an image processing 
10 apparatus according to the present invention. 

Fig. 2 is a diagram to describe in further detail the process of Fig. 1. 
Fig. 3 is a block diagram of an image processing apparatus according 
to a first embodiment of the present invention. 

Fig. 4 is a flow chart of the process of a matrix operation unit 209 of 

15 Fig. 3. 

Fig. 5 is a flow chart of the process carried out by the matrix 
operation unit of an image processing apparatus according to a second 
embodiment of the present invention. 

Fig. 6 is a block diagram of an image processing apparatus according 
20 to a third embodiment of the present invention. 

Fig. 7 is a flow chart of a process of a matrix operation unit 209 of 

Fig. 6. 

Fig. 8 is a block diagram showing the structure of an image 
processing apparatus according to a fourth embodiment of the present 
25 invention. 

Fig. 9 is a flow chart of a process of a matrix operation unit 309 of 

Fig. 8. 

Fig. 10 is a diagram to describe the advantage of the image 
processing apparatus of the present invention. 
30 Fig. 11 is a block diagram showing an example of an image 

processing apparatus formed of only a synchronous circuit. 
DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Referring to Fig. 1, an image process of light load is carried out on 
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the entire region of document image data 100 by means of a synchronous 
type circuit. The result is wiitten into a memory. An asynchronous 
circuit carries out a process such as matrix operation on regions (for 
example, text region) 100a and 100b in document image data 100 that 
5 require processing of heavy load. The processed result of the 

asynchronous type circuit is overwritten on the memory in which the 
processed result of the synchronous type circuit has been written. 

The synchronous type circuit carries out high speed image processing 
on the entire region of the document image data whereas the asynchronous 
10 type circuit carries out image processing exclusively on the region that 

requires heavy load processing such as a matrix operation. The process 
carried out by the synchronous type circuit includes the one-dimensional 
process such as y correction. The process carried out by the asynchronous 
yi circuit includes the two-dimensional process such as edge enhancement 

f U 15 process on a text region and smoothing process on a halftone dot region. 
J* Here, a synchronous circuit refers to a hardware circuit such as an 

jU ASIC. An asynchronous circuit refers to a circuit that is formed of a MPU 

it and the like carrying out a process with software, 

[j By virtue of the flexibility of the asynchronous type circuit, the 

w 20 processing data can be modified easily when the resolution or the like of the 
™ image data is altered. Therefore, increase in cost can be suppressed. 

In other words, the software of the asynchronous type circuit is 
formed in a rewritable manner. 

Fig. 2 is a diagram to describe in further detail the operation of the 
25 image processing apparatus. Referring to Fig. 2, the image of interest is 
stored in an input image memory having the capacity of WX pixels 
horizontally and WY pixels vertically. Data IDi of one pixel in the input 
image memory is read out in series and subjected to y correction with a 
predetermined function to become y corrected data ODil. 
30 y corrected data ODil is recorded at a corresponding address in the 

output image memory, y correction is carried out on all the pixels. 

When a pixel of interest (target pixel) in the input image memory 
resides in a region of interest that requires matrix operation, a matrix 
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operation is carried out on the target pixel. It is assumed that the address 
of the target pixel is (X, Y), and data of that address is D (X, Y). The 
matrix, i.e. the mask region of the filter process, has the size of M in the X 
direction and N in the Y direction, centered about the target pixel. The 
5 mask is k (i, j). Data D' (X, Y) subjected to the matrix operation is 
represented by the following equation. 

D'(X, Y) = TZ[k(i,j)D(x-(M- l)/2 + i, Y-(N- 1) / 2 + j) ] 

10 where T, k (i, j) are constants, and i = 0, M - 1; j = 0, N - 1. Then, y 
^ correction is carried out on this data to result in data ODi2. This process 

lz is carried out on all the pixels in the region of interest of the matrix 

!ff operation. 

^3 When image data ofMxN centered about the target pixel is read out 

f 1J 15 from the input image memory and readdressed, the above equation can be 
presented as: 

j: D' (X, Y) = T Z [ k (i, j) D (i, j) ] (1) 

LJ i = 0, ..,M-l;j = 0, ...,N - 1 

^3 20 

First, y corrected data ODil is written into the output image memory, 
and then data ODi2 subjected to the matrix operation and y correction is 
written into a predetermined region. 

First Embodiment 

25 Fig. 3 is a block diagram showing a structure of an image processing 

apparatus according to a first embodiment of the present invention. 
Referring to Fig. 3, the image processing apparatus includes an input 
image memory 201, an input memory control unit 203, a y correction unit 
205, an output memory control unit 207, a matrix operation unit 209, and 

30 an output image memory 211. For the process of input data, a data bus for 
input data is used. For the process of the output data, a data bus for 
output data is used. 

Image data input by means of a scanner and the like is stored in 
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input image memory 201. The storage capacity of input image memory 
201 corresponds to WX x WYfor a monochromatic image, corresponds to 
WX x WY x 3 for a color (RGB) image, and corresponds to WX x WY x 4 for 
a color (CMYK) image, when the size of the image data is (number of main 
scanning pixels, number of sub scanning pixels) = (WX, WY). For the sake 
of simplification, it is assumed that the storage capacity of input image 
memory 20 1 is WX x WY in the following. Although image data coding is 
not effected in the present embodiment, the memory capacity of input 
image memory 201 can be reduced by carrying out a coding process. 

Input memory control unit 203, y correction unit 205 and output 
memory control unit 207 are implemented by an ASIC circuit (synchronous 
type circuit) that allows high speed processing since the circuits are formed 
of relatively simple logics. Matrix operation unit 209 is formed of a MPU 
(asynchronous type circuit) that allows complicated operation to read out 
image data ofMxN pixels and apply a complicated operation thereon. 

Under DMA (direct memory access) control, input memory control 
unit 203 reads out image data stored in input image memory 201 
sequentially from the start address to the end address (start address + WX 
x WY). Input memory control unit 203 includes an internal register in 
which a status flag (imb flag) is held indicating whether access to input 
image memory 201 is busy or not. 

y correction unit 205 converts and outputs the input data according 
to a table of output data (8 bits: 0 - 255) corresponding to the input data (8 
bits: 0 - 255). 

Under DMA control, output memory control unit 207 sequentially 
writes the y-corrected data from the start address up to the end address 
(start address + WX x WY) into output image memory 211. Output 
memory control unit 207 includes an internal register in which a status flag 
(omb flag) is held indicating whether access to output image memory 211 is 
busy or not. 

Matrix operation unit 209 monitors the imb flag of input memory 
control unit 203 to read out the data of M x N pixels including the target 
pixel by random access from input image memory 201 when the imb flag 




indicates a non-busy state. Then, the matrix operation such as edge 
enhancement is carried out on the text region and smoothing on the 
halftone region. Matrix operation unit 206 further carries out a process 
likewise the y correction formed of logic. Matrix operation unit 209 
5 monitors the omb flag of output memory control unit 207 to write the 

resultant data of the matrix operation into the address of output image 
memory 211 corresponding to the target pixel when the omb flag indicates 
a non-busy state. Accordingly, y-corrected data from y correction unit 205 
is replaced in output image memory 211. 

10 The region on which matrix operation is to be carried out can be 

determined by matrix operation unit 209. Alternatively, a region 
determined in advance can be input through the data bus. 

The output image data subjected to y correction and matrix operation 
in output image memory 211 is provided to a printer engine and the like 

15 through the data bus. 

When image data is written into input image memory 201 in the 
present embodiment, input memory control unit 203, y correction unit 205 
and output memory control unit 207 carry out a one-dimensional process of 
light load such as y correction on the image data in input image memory 

20 201. Since the imb flag in input memory control unit 203 indicates a busy 
state during this process, matrix operation unit 209 cannot read out data 
from input image memory 201. 

Following the y correction on all the image data in input image 
memory 201, matrix operation unit 209 extracts image data that is the 

25 subject of processing from a predetermined region of the image data stored 
in input image memory 201. A two-dimensional process is applied on the 
extracted image data. The processed result is overwritten on the 
predetermined region in output image memory 211. 

Fig. 4 is a flow chart for describing the process carried out by matrix 

30 operation unit 209. Referring to Fig. 4, initialization is carried out at step 
S 10 1. At S 103, the address of the target pixel that becomes the subject of 
processing in input image memory 201 is determined. At step S105, the 
access status (imb flag) of input memory control unit 203 is confirmed. 
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Waiting is conducted when the imb flag indicates a busy state (S 107). 

When the imb flag indicates a non-busy state, control proceeds to 
step S109 where image data of the matrix ofMxN centered about the 
target pixel is read out. At step Sill, the operation process of the MxN 
5 matrix is carried out. 

At step SI 13, y correction is carried out on the image data subjected 
to the operation process. 

At step SI 15, the access state (omb flag) of output memory control 
unit 207 is confirmed. At step SI 17, waiting is conducted until the omb 
10 flag of output memory control unit 207 indicates a non-busy state. At step 
SI 19, the write address of output image memory 211 is confirmed by output 
memory control unit 207. This causes the wilting of the data subjected to 
matrix operation to attain a standby state until y-corrected data is written 
into the address of the target pixel by output memory control unit 207. At 
fU 15 step S121, the data subjected to matrix operation is overwritten in output 
" image memory 211. 

i* At step S123, determination is made whether the matrix operation 

[j; has been completed for all the pixels in the region of interest for the matrix 

i 1 operation. The routine ends when YES. When NO at step S123, control 

^3 20 returns to step S103. 

hJ According to the above-described steps of S 105 and 107, matrix 

operation unit 209 reads out the image data from input image memory 201 
when input memory control unit 203 is not busy. Accordingly, the access 
of input image memory 201 by input memory control unit 203 is given 
25 priority. 

By the above-described steps of Si 15 and Si 17, the writing operation 
by matrix operation unit 209 is carried out when output memory control 
unit 207 is not busy. Accordingly, access to output image memory 2 1 1 by 
output memory control unit 207 is given priority. A memory storing the 
30 processed result of one line can be provided in output memory control unit 
207. Output memory control unit 207 may access output image memory 
211 only when the process of one line has been completed by output 
memory control unit 207 and the process of the next line has not yet been 
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initiated. 

Second Embodiment 

The image processing apparatus according to the second embodiment 
of the present invention has a structure identical to that of the first 
5 embodiment. Therefore, description thereof will not be repeated. The 
image processing apparatus of the second embodiment executes the flow 
chart represented in Fig. 5 instead of the process represented in Fig. 4. 
The image processing apparatus of the present embodiment is 
characterized in that a halftone region or an edge region is detected and 

10 image processing carried out on the detected region by scanning the image 
data stored in input image memory 20 1. 

Referring to Fig. 5, initialization is carried out at step S201. At step 
S203, determination is made whether input image memory 201 can be 
accessed or not by referring to the imb flag. Waiting is conducted until 

15 access is allowed. 

At step S205, a matrix of 5 x 5 pixels centered about the target pixel 
is extracted from the image data stored in input image memory 201. At 
step S207, determination is made whether the target pixel is an isolated 
point or not. When YES at S207, control proceeds to step S209 where a 

20 larger matrix of 9 x 41 pixels is extracted, and then determination is made 
whether that matrix corresponds to a halftone image or not (S211). 

When YES at S211, control proceeds to step S213 to carry out a 
smoothing process using the data of the 5x5 pixels. Then, control 
proceeds to step S215 to carry out a y correction. At step S217, the omb 

25 flag is confirmed to determine whether access to output image memory 211 
is allowed or not. When access to output image memory 211 is allowed, 
control proceeds to step S219 to write the data into output image memory 
211. At step S221, determination is made whether the process has been 
completed for all the data. When YES at step S221, the routine ends. 

30 When NO at step S221, control returns to step S203. Also, when 

NO at step S221, control proceeds to step S221. 

When determination is made that the target pixel is not an isolated 
point at step S207, control proceeds to step S223 to determine whether the 
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subject is an edge image. When YES, control proceeds to step S225 to 
carry out edge enhancement. Then, control proceeds to step S215. 

When NO at step S223, control proceeds to step S221. 

The processing time by the matrix operation unit according to the 
present embodiment is set forth in the following. 

(processing time of target pixel when corresponding to halftone dot) = 
(matrix read out time of 5 x 5 pixels) + (time for determining an isolated 
point) + (matrix read out time of 9 x 41 pixels) + (time for determining 
halftone dot) + (time for smoothing process of 5 x 5 pixels) + (time for y 
correction) 

(processing time of target pixel when corresponding to edge) = 
(matrix read out time of 5 x 5 pixels) + (time for determining an isolated 
point) + (time for edge enhancement process of 5 x 5 pixels) + (time for y 
correction) 

(processing time of target pixel when not corresponding to edge or 
halftone dot, and is not an isolated point) = (matrix read out time of 5 x 5 
pixels) + (time for determining an isolated point) 

(processing time of target pixel when not corresponding to edge or 
halftone dot, and is an isolated point) = (matrix read out time of 5 x 5 
pixels) + (time for determining an isolated point) + (matrix read out time of 
9x41 pixels) + (time for determining a halftone dot) 

In view of the foregoing, the relationship of the processing time is 
established as follows: 

(processing time of target pixel when not corresponding to edge or 
halftone dot, and is not an isolated point) < (processing time of target pixel 
when corresponding to edge) < (processing time of target pixel when not 
corresponding to edge or halftone dot, and is an isolated point) < 
(processing time of target pixel corresponding to halftone dot). 
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Third Embodiment 

Fig. 6 is a block diagram showing a structure of an image processing 
apparatus according to a third embodiment of the present invention. The 
image processing apparatus of the third embodiment corresponds to the 
5 image processing apparatus of the first embodiment shown in Fig. 3 with a 
region determination memory 251 connected to the data bus. 
Determination of a region of the image data stored in input image memory 
201 is recorded in advance in region determination memory 251. In the 
present embodiment, region determination memory 251 distinguishes 

10 whether one pixel of the image data stored in input image memory 201 

corresponds to an edge region, a halftone region, or a region other than the 
edge or halftone region. 

Fig. 7 is a flow chart of the process carried out by matrix operation 
unit 209 of the third embodiment. 

15 At step S301, initialization is carried out. At step S303, the address 

of the target pixel is extracted. 

At step S305, determination is made whether the target pixel 
corresponds to an edge region or a halftone region by region determination 
memory 251. When the target pixel corresponds to an edge or halftone 

20 region, control proceeds to step S307 to determine whether access to input 
image memory 201 is allowed or not through the imb flag. Waiting is 
conducted until access is allowed. At step S309, the data of the matrix of 5 
x 5 pixels around the target pixel is read out. A smoothing process is 
carried out if the target pixel corresponds to a halftone region (S3 11). An 

25 edge enhancement process is carried out if the target pixel corresponds to 
an edge region (S321). Then, y correction is carried out on the data of the 
target pixel (S3 13). Next, determination is made whether output image 
memory 211 can be accessed or not through the omb flag. Waiting is 
conducted until access is allowed (S3 15). 

30 At step S3 17, image data is written into output image memory 211. 

At step S3 19, determination is made whether the process of all the data has 
been completed or not. When YES at step S3 19, the main routine ends. 
When NO at step S3 19, control returns to step S303. When NO at 
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S305, control proceeds to step 319. 

The processing time by the matrix operation unit in the third 
embodiment is set forth in the following. 

(processing time of target of interest when corresponding to a 
5 halftone dot) = (time for reading out determination of target pixel) + (read 
out time of 5 x 5 pixels) + (time for smoothing process of 5 x 5 pixels) + 
(time for y process) 

(processing time of target pixel when corresponding to edge) = (time 
10 for reading out determination result of target pixel) + (read out time of 5 x 
5 pixels) + (processing time for edge enhancement of 5 x 5 pixels) + (time 
for y correction) 

(processing time of target pixel when not corresponding to edge or 
15 halftone dot) = (time for reading out determination result of target pixel) 

In view of the foregoing, the relationship of the processing time is 
established as follows: 

(processing time of target pixel when not corresponding to edge or 
20 halftone dot) < (processing time of target pixel when corresponding to 

halftone dot) < (processing time of target pixel when corresponding to edge) 

Fourth Embodiment 

Fig. 8 is a block diagram showing a structure of an image processing 
25 apparatus according to a fourth embodiment of the present invention. The 
image processing apparatus of the present embodiment differs from the 
image processing apparatus of Fig. 3 in that input image memory 301 and 
output image memory 311 are set as dual port memories. Accordingly, the 
process by the synchronous type circuit and the process by the 
30 asynchronous type circuit can be carried out in parallel to further increase 
the image processing speed. 

Fig. 9 is a flow chart of the process carried out by a matrix operation 
unit 309 according to the image processing apparatus of the fourth 
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embodiment. The flow chart of Fig. 9 differs from the flow chart of Fig. 4 
in that the step of identifying the access status of the input image memory 
and the output image memory through the imb flag and the omb flag (steps 
S105, S107, SI 15 and Si 17 of Fig. 4) is deleted. This is because matrix 
5 operation unit 309 can access the memory any time without taking account 
of the access to the memory in the fourth embodiment. 

It is to be noted that the output result of matrix operation unit 309 
must be overwritten after the output result of the synchronous type circuit 
is written into output image memory 311. Therefore, control is provided so 
10 that the writing status into output image memory 31 1 by output memory 
C3 control unit 307 is identified at step S419 of Fig. 9 to allow the operation 

% i result of matrix operation unit 309 to be written into output image memory 

Id 311 after the output result from the synchronous type circuit is written. 

2 ^ Fig. 10 is a diagram to describe the advantage of the image 

f y 15 processing apparatus of the present invention. It is assumed that the 
^ relative ratio of the image processing time is y correction (one-dimensional 

L process): text correction (5x5 matrix operation) =1:25 and one page of 

M image data is formed of 100 pixels. When the entire process is carried out 

" by a synchronous type circuit, i.e. when matrix operation is carried out with 

\2 20 the synchronous type circuit on all the regions, the processing time of one 
^3 page is 100 (pixels) x 25. When only the y correction is carried out by the 

synchronous type circuit as in the above embodiments, i.e. when matrix 
operation is carried out with the asynchronous type circuit only on a 
particular region (text distributed region), the processing time is 50 (pixels) 
25 x 25 and 25 (pixels) x 25 in the case where the text is distributed for 50% 
and 25% in one page, respectively. 

In other words, the processing time becomes longer when the all the 
process is carried out by a synchronous type circuit since text correction (5 x 
5 matrix operation) is carried out on all the pixels irrespective of the text 
30 distribution. In contrast, only the required portion is subjected to text 

correction by virtue of the usage of both a synchronous type circuit and an 
asynchronous type circuit as in the above embodiments. Therefore, the 
processing speed is improved. 
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In the above embodiments, the processes are executed within a 
unitary apparatus. However, the processes can be carried out by a 
plurality of apparatuses. For example, the synchronous processing unit 
and the asynchronous processing unit can be implemented by separate 
apparatuses. 

A program for executing the above process (S101 - S123) stored in a 
storage medium can be loaded into the matrix operation unit and executed 
therewith. 

Although the presentinvention has been described and illustrated in 
detail, it is clearly understood that the same is by way of illustration and 
example only and is not to be taken by way of limitation, the spirit and 
scope of the present invention being limited only by the terms of the 
appended claims. 
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